Papers with Variational Autoencoder
Distantly Supervised Relation Extraction with Sentence Reconstruction and Knowledge Base Priors (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to facilitate distantly supervised relation extraction are noisy instances, long-tail relations and unbalanced bag sizes. |
| Approach: | They propose a multi-task approach to facilitate distantly supervised relation extraction by bringing closer the representations of sentences that contain the same Knowledge Base pairs. |
| Outcome: | The proposed approach improves performance on two datasets created via distant supervision. |
Non-Autoregressive Neural Machine Translation with Consistency Regularization Optimized Variational Framework (2022.naacl-main)
Copied to clipboard
| Challenge: | Variational Autoencoder (VAE) is an effective framework to model the interdependency for non-autoregressive neural machine translation (NAT). |
| Approach: | They propose to use Variational Autoencoder to model interdependency for non-autoregressive neural machine translation (NAT) a posterior consistency regularization approach is proposed to improve translation quality . |
| Outcome: | The proposed model is 1.5/0.7 and 0.8/0.3 BLEU points faster than the baseline model. |
Learning Disentangled Representations for Natural Language Definitions (2023.findings-eacl)
Copied to clipboard
| Challenge: | Disentangling the encodings of neural models is a fundamental aspect for improving interpretability, semantic control and downstream task performance in Natural Language Processing. |
| Approach: | They propose to use syntactic and semantic regularities in textual data to provide models with both structural biases and generative factors. |
| Outcome: | The proposed model outperforms baselines on several qualitative and quantitative benchmarks and improves the results in the downstream task of definition modeling. |
Improving Variational Autoencoder for Text Modelling with Timestep-Wise Regularisation (2020.coling-main)
Copied to clipboard
| Challenge: | Variational Autoencoders (VAEs) have been widely used in text modelling but posterior collapse is a problem when RNN-based models are employed. |
| Approach: | They propose a timestep-wise regularisation VAE architecture which can effectively avoid posterior collapse when used in text modelling. |
| Outcome: | The proposed model avoids posterior collapse and can be applied to any RNN-based VAE model. |
A Batch Normalized Inference Network Keeps the KL Vanishing Away (2020.acl-main)
Copied to clipboard
| Challenge: | Variational Autoencoder (VAE) is widely used to approximate a model’s posterior on latent variables. |
| Approach: | They propose to let the Kullback–Leibler divergence individual follow a distribution across the whole dataset and analyze that it is sufficient to prevent posterior collapse by keeping the expectation of the KL’s distribution positive. |
| Outcome: | The proposed approach can avoid posterior collapse effectively and efficiently without introducing any new model component or modifying the objective. |
Learning Disentangled Latent Topics for Twitter Rumour Veracity Classification (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to rumour veracity classification relied on feature engineering. |
| Approach: | They propose a model which disentangles the informational content of a tweet from the manner in which it is written. |
| Outcome: | The proposed model disentangles the informational content of a tweet from the manner in which the information is written. |
Do sequence-to-sequence VAEs learn global features of sentences? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Autoregressive language models are often trained without explicit conditioning labels . authors question claim that latent vectors can capture global features in unsupervised manner . |
| Approach: | They propose to use a sequence-to-sequence architecture to learn latent variables . they find that VAEs are prone to memorizing the first words and sentence length . |
| Outcome: | The proposed model is prone to memorizing the first words and sentence length, the authors show . et al., 2016: a new model learns latent variables that are more global, more predictive of topic or topic labels . authors question this claim, say it is a waste of time and money . |
A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text (D19-1)
Copied to clipboard
| Challenge: | Variational Autoencoders are powerful language models and effective representation learning frameworks. |
| Approach: | They propose a fix for posterior collapse which improves held-out likelihood, reconstruction and latent representation learning . |
| Outcome: | The proposed fix significantly improves held-out likelihood, reconstruction, and latent representation learning compared with previous state-of-the-art methods. |
Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for language understanding and understanding can be trained to provide contextualized representations of words based on text data. |
| Approach: | They propose a large-scale language VAE model Optimus that is pre-trained on large text corpus and fine-tuned for various language generation and understanding tasks. |
| Outcome: | The proposed model achieves new state-of-the-art on VAE language modeling benchmarks. |
VaeDiff-DocRE: End-to-end Data Augmentation Framework for Document-level Relation Extraction (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for Document-level Relation Extraction assume a uniform label distribution, resulting in suboptimal performance on real-world, imbalanced datasets. |
| Approach: | They propose a method that leverages the Variational Autoencoder architecture to capture all relation-wise distributions formed by entity pair representations and augment data for underrepresented relations. |
| Outcome: | The proposed method outperforms state-of-the-art models on two benchmark datasets and is available on github. |
Learning Disentangled Representations of Negation and Uncertainty (2022.acl-long)
Copied to clipboard
| Challenge: | Negation and uncertainty modeling are long-standing tasks in natural language processing. |
| Approach: | They propose to disentangle negation, uncertainty, and content using a Variational Autoencoder by supervising latent representations using auxiliary objectives. |
| Outcome: | The proposed model can disentangle negation, uncertainty, and content using a Variational Autoencoder. |
Unsupervised Opinion Summarisation in the Wasserstein Space (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on opinion summarisation of social media posts has focused on reviews . however, it is important to capture user opinions in online discussions over specific topics . |
| Approach: | They propose an unsupervised opinion summarisation model which uses the Wasserstein distance to generate a single summary from a group of documents. |
| Outcome: | The proposed model outperforms the state-of-the-art on ROUGE metrics and produces the best summaries with respect to meaning preservation according to human evaluations. |
Explainable Disentangled Representation Learning for Generalizable Authorship Attribution in the Era of Generative AI (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods struggle with content-style entanglement, leading to poor generalization across domains. |
| Approach: | They propose an explanation-by-design framework that explicitly disentangles style from content through architectural separation-by design. |
| Outcome: | The proposed framework disentangles style from content through architectural separation-by-design. |